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Summary of Review 


In this report, a school choice advocacy group presents results from its survey of K-12 par- 
ents within and across the public and private sectors. They report that parents are highly 
satisfied with voucher and tax credit scholarship programs and suggest that the findings 
support the expansion of school choice programs. However, these and other findings are 
consistent with research showing general parental satisfaction with their children’s schools. 
One underemphasized finding is that substantial proportions of public school families—the 
largest constituency of K-12 parents in Indiana—did not participate in private school choice 
programs because they are happy with their current schools and want to support public 
schools. The survey and analysis fall short in four ways. First, three incompatible data col- 
lection methods were used to collect small samples of non-representative groups of Indiana 
parents. Second, the statistical analyses are too weak to draw clear conclusions. Third, while 
organized like a conventional research study, the report appears to be designed to advance 
an agenda rather than provide substantive answers to important policy questions. Finally, 
the report provides little new information about parents’ experiences with their children’s 
schools. Thus, the report does not add to our knowledge about school choice in Indiana or 
provide much useful information about public support for school choice programs. 
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I. Introduction 


In Why Indiana Parents Choose, Andrew C. Catt and Evan Rhinesmith’ attempt to broaden 
policy discussions about school choice beyond student achievement by surveying parents 
across the public and private school sectors in Indiana about their knowledge of private 
school choice options, why they chose their children’s schools, the difficulties parents en- 
countered finding schools, parents’ satisfaction with their children’s schools, and their goals 
for their children’s education. Five groups of parents participated in the survey: parents 
whose children attend a school operated by a public school district, public charter school 
parents, voucher parents, tax-credit scholarship parents, and non-choice private school par- 
ents, or parents who pay for their children’s private school tuition without state support. 


Since 2011, Indiana has operated a statewide voucher program, the School Choice Scholar- 
ship Program, which funds students’ tuition and fees at participating private schools. Since 
2010, Indiana has also provided a 50% tax credit for individuals and corporations who do- 
nate to a scholarship granting organization (SGO) that awards private school scholarships.’ 
The School Choice Scholarship Program is one of the largest in the country. In the 2016-17 
academic year, 34,299 or 2.9% of Indiana’s K-12 students participated in the program. Fif- 
ty-five percent of these students had not previously attended an Indiana public school before 
participating in the School Choice Scholarship Program. As enrollment in these programs 
increases, the state funds directed to these programs has also risen. In 2016-17, Indiana dis- 
tributed $146 million in Choice Scholarships to participating students.° 


Indiana is a good setting to compare parents across the public and private school sectors giv- 
en the relatively high participation rates in its private school choice programs and the size 
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of the charter school sector. According to estimates provided in the report, 3.2% of K-12 stu- 
dents in the state attend charter schools, and 5.2% are non-choice private school students. 


II. Findings and Conclusions 


There are three major sections of the report. The first is focused on comparing parents’ ex- 
periences with Indiana’s private school choice programs, the Choice Scholarship Program 
(voucher families) and the School Scholarship Tax Credit Program (tax credit families). The 
second compared the five groups of parents on their satisfaction with their schools and the 
factors they prioritized in enrolling at their children’s schools. The final section of the report 
compared non-choice private school parents, district school parents, and charter school par- 
ents who reported that their children changed schools, and the factors that influenced that 
decision. 


Experiences of private school choice families 


Both voucher and tax credit scholarship families reported high satisfaction with the private 
school choice programs in which they they participated (83% and 85%, respectively). Urban 
and suburban parents were more satisfied with the Choice Scholarship Program than small 
town and rural parents. Likewise, parents with a college degree or higher were also more 
satisfied with the Choice Scholarship Program than parents without a college degree. 


The vast majority of families (85%) who participate in a private school choice program con- 
tinue to participate in subsequent school years. Most of the families who leave the program 
do so because their families are no longer eligible. They either do not meet the income re- 
quirements or their child graduated from high school. 


The parents from the other sectors (non-choice private school, district school, and charter 
school parents) were asked to select the reasons they did not participate in the private school 
choice programs. Thirty-five percent of district school families and 42% of charter school 
families reported that they were happy with the schools their children attend. Finally, while 
22% of district school families reported that they wanted to support public schools, only 4% 
of non-choice private school families and 7% of charter school families selected this option. 


Parents’ satisfaction with schools and priorities for choosing their 
children’s schools 


The majority of parents across the five sectors (74% or more) were somewhat satisfied or 
completely satisfied with their children’s schools. The major differences between the groups 
were in the balance between the “somewhat satisfied” and “completely satisfied” responses. 
For example, district school parents were evenly split between somewhat satisfied and com- 
pletely satisfied, whereas larger percentages of parents participating in choice (public and 
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private) selected completely satisfied. 


The five groups of parents were also asked to identify the most important reason why they 
chose their children’s schools. Forty-five percent of district school parents chose a school 
because it was their assigned district school. District school parents also prioritized a loca- 
tion close to home and work (19%) and academics (11%). The top choice for voucher (23%) 
and non-choice private school families (30%) was religious environment and instruction, 
followed by academics and moral, character, and values instruction. The top choice among 
tax credit scholarship families and charter school families was academics (24% and 19%, 
respectively). However, the second most selected feature among tax credit scholarship fami- 
lies was religious environment and instruction, while for charter school families it was small 
school. Across all five sectors, academics were among the top three most important reasons 
families chose their schools. 


Parents were also asked how they supported their children’s education and their goals for 
their children’s education. At least 80% of district school, non-choice private school and 
charter school parents reported that they helped their children with homework at least one 
night per week. Compared to the other three groups, voucher and tax credit scholarship par- 
ents reported helping with homework at lower rates (68% and 62%, respectively). 


In general, there were few differences in how parents across the five sectors ranked 19 edu- 
cational goals. For example, between 85 and 95% of parents in each sector reported that it 
was very important or extremely important that their children develop strong critical think- 
ing skills, and learn good study habits and self-discipline. Likewise between 80 and 90% of 
all parents responded that it was very important or extremely important that their children 
can identify their own interests and pursue their talents on their own. The largest range in 
parents’ responses across the five sectors was on the goal of finishing high school with job 
skills that do not require further education. 


Parents who changed schools 


Substantial proportions of non-choice private (30%), district school (39%), and charter 
school parents (47%) reported that their children changed schools during their educational 
careers. Most of these parents moved their children from district schools. Seventy-seven 
percent of district school parents moved their children from another district school, com- 
pared to 64% of charter school parents, and 46% of non-choice private school parents. 


III. The Report’s Rationale for Its Findings and Conclusion 


The results presented in the report are drawn from a descriptive analysis of the survey ques- 
tions. Parents were divided into groups by the sector of the school their children attended. 
The responses of parents within each sector to individual survey questions were compared 
by locale, race, income, and education. The report paired subgroups and used z-tests to as- 
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sess statistically significant differences in the proportions of each subgroup that selected a 
response category on a survey question. 


IV. The Report’s Use of the Research Literature 


While a cursory search on Google Scholar using the search terms “parent satisfaction” and 
“school choice” yields over a thousand articles, the literature review section is less than three 
pages of a 60-page report and tended to emphasize non-refereed works from the University 
of Arkansas and EdChoice. The report did not claim that the review is comprehensive; rather 
it is intended to focus on recent research. The earliest study included in the review was pub- 
lished in 2001. Most studies suggest that parents who choose their children’s schools tend to 
be more satisfied than those who do not. However, many of the studies the report reviewed 
compared groups of parents who are active choosers: parents who participate in lotteries for 
scholarships or attendance slots at oversubscribed schools, and parents who attend different 
types of charter schools. Active choosers tend to be more positive about their school choice 
decisions. 


V. Review of the Report’s Methods 


In survey research, the validity and utility of a study is dependent on the methodological 
rigor of the research design. This is where this study diverged sharply from conventional 
standards for conducting research. There are significant problems with both the methods 
used to conduct the survey and the data analysis. 


The study utilized three sampling methods and did not provide a rationale for combining 
them, which is highly irregular. In the first phase of data collection, a private research firm 
administered an online survey to a group of subjects they identified using proprietary meth- 
ods; the response rate from this phase was extremely low (1.6%). The second and third 
phases of data collection targeted school choice parents. In the second phase, parents who 
participated in EdChoice’s 2013 and 2016 surveys of private school parents were contacted. 
Slightly more than 5,000 parents were surveyed and 774 (15%) of these responded with com- 
plete (545) or partial surveys (229). In the third phase, charter school parents were surveyed 
using a snowball sampling technique. Eighty-two charter school leaders were contacted by 
a charter advocacy organization and asked to send the survey to their schools’ parents. This 
strategy yielded 89 additional completed and partial surveys. 


The use of the latter two methods is problematic because they are clearly non-random sam- 
pling procedures. While the report acknowledged this limitation, it downplayed: a) the ad 
hoc methods of data collection; and b) the extent to which the choice parents are overrep- 
resented in the sample and were a self-selected group. More than half of the private school 
parents were past respondents in EdChoice’s surveys of private school parents. Similarly, 
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just under half of the charter school parents were asked to participate in the survey by their 
school principals. These more personal connections may have made these groups of parents 
more likely to participate in the survey. Notably, the highest response rate (15%) was among 
private school parents who were prior participants in EdChoice surveys; this group also re- 
ceived six reminder emails. In the first phase of data collection, parents were sent only two 
reminders to complete the survey. As noted above, the response rate from the email survey 
of all parents in the first phase of data collection was substantially lower. 


Similarly, because the email addresses used in the first phase of the survey administration 
were collected by a private research firm using undisclosed proprietary methods, it is not 
clear how parents were identified, and the extent to which parents whose children attended 
district schools were targeted. Only a small fraction of district school parents participated in 
the survey (0.14% of the population). While the response rates of the other groups of parents 
were also small relative to their population sizes, they were substantially higher than the re- 
sponse rates of district school parents. Finally, the research firm that conducted the survey 
dropped 960 responses to the survey for “suspicion of specious responses” which reduced 
the final analysis sample to 3,532 parents.5 


After a series of questions asking parents about their participation in private school choice, 
programs, the core of the survey consisted of Likert items asking parents how satisfied they 
were with their children’s schools and, for private school parents, the private school choice 
program they were participating in. Another pair of questions asked parents to identify the 
factors that influenced their choice of their children’s current 
While the problems _ schools and then to select the most influential factor from that 
with the sample are list. Non-choice private school, district school, and charter 
substantial, the data school parents were asked why they did not participate in the 
and anaylses are private school choice program. Parents with more than one 
child enrolled in a K-12 school were directed to answer the 
survey based on their experiences with their oldest child. Par- 
ents whose children were no longer in school but had a child enrolled in school within the 
last five years were asked to answer the survey for their most recently enrolled child. The 
survey itself provides little new or useful information. For example, the question asking 
parents to identify all of the factors that influenced their decisions to have their children 
attend their current schools in a yes/no format resulted in an unwieldy list of factors that is 
difficult to interpret. A substantial proportion of the district school parents selected “This is 
my assigned neighborhood school,” which is not surprising since that response is targeted to 
that group, which makes it difficult to meaningfully compare their responses to the parents 
from the other sectors. 


also problematic 


While the problems with the sample are substantial, the data and analyses are also problem- 
atic. The report provides a descriptive analysis of the survey questions. Parents were divided 
into groups by the sector of the schools their children attended. Parents’ responses were also 
compared by subgroups within each sector based on locale, race, household income and ed- 
ucation. Statistically significant differences were noted where relevant. The report explicitly 
stated that causality should not be inferred from any of the findings. These are all standard 
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research practices. However the report should have been more transparent in how some of 
the key findings were presented, and in particular how non-response on key questions was 
addressed in the analysis. The report also should have more fully addressed the extent to 
which the respondents were representative of the families attending each type of school. In 
addition, the survey did not contain questions asking parents about their satisfaction with 
specific aspects of their children’s current schools. A final issue is that the statistical pro- 
cedures and findings are not clearly and directly reported. I discuss each of these in turn 
below. 


Non-response on key questions 


In the description of the methodology and data sources, the report noted that a substantial 
number of parents from each sector submitted partial responses to the survey. While in the 
report, the “raw response” rates were reported underneath the figures reporting the findings 
on individual questions (p. 13), this does not address the large number of missing responses 
on key questions, which were not accounted for in the analysis. For example, the first sets 
of analyses focus on voucher and tax-credit scholarship families and report the number 
of years families have participated in school choice programs and their satisfaction in the 
school choice program. The numbers underneath each figure indicate there were non-re- 
sponses of 17% and 14% for voucher and tax-credit parents, respectively, which are substan- 
tial. These questions have some of the highest non-response rates for these two groups of 
families. In comparison, 3% of the voucher parents and 4% of the tax-credit parents did not 
respond to the questions about ease of finding a private school and about their satisfaction 
with their children’s current schools. If the missing responses are incorporated into the 
analysis, the number of voucher parents from the full sample of 733 parents who reported 
being completely satisfied with Indiana’s school choice programs drops from 62% to 51%, 
which is a substantial difference, although still a majority of parents. That said, it is rather 
unsurprising that parents are largely satisfied with these programs because they subsidize 
the cost of private school attendance for a small group of families. Three percent of K-12 
students in Indiana attend private schools using the voucher provided by the Choice Schol- 
arship Program. Given these issues, these findings should not be overemphasized by virtue 
of their placement as the first finding reported in the Executive Summary and the second 
in the body of the report after an analysis of years of participation in the voucher program. 


Perhaps more importantly, the background questions asking parents about their race and 
income were among the questions with the most missing information. For example, based 
on the information reported in the tables provided in Appendix 2, 21% of the voucher par- 
ents did not respond to the question about household income (p. 44). While there were little 
differences in satisfaction among parents who provided information about their household 
income, we do not know how parents who did not answer the income question responded. 
Likewise, because the question about satisfaction and private school choice programs had 
a relatively high non-response rate, only 65% of the voucher parents in the analysis sample 
were included in the subgroup analysis of the relationship between income and satisfaction. 


The non-response rates for the questions asking parents to report their race, income, and 
education were similarly high across groups of parents. For example, the supplemental in- 
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formation about the questionnaire indicated that 27% of the parents whose children at- 
tended schools operated by school districts were missing information on the questions ask- 
ing them to report their race/ethnicity and household income.* Without knowing how the 
parents who did not provide information on the background question answered the survey 
questions about the choice programs, it is difficult to assess whether or not this loss of cases 
was consequential. While the report noted that the subgroup analyses should be interpreted 
with “strong caution” because of small sample sizes, the report should have reported missing 
values more directly and accounted for them in the analyses when relevant. 


Representativeness of the samples 


While the survey collected information about families’ backgrounds, and the analysis pro- 
vided comparisons across subgroups, the report did not address the extent to which the 
samples for each sector are representative samples, aside from a brief disclaimer. This is a 
striking omission given that this information is readily available from state reports and data. 
For example, comparing the results provided in the supplemental information about the 
survey’ with data and reports available from the Indiana Department of Education® suggests 
that White parents are overrepresented and Hispanic parents are underrepresented in the 
voucher and district school samples, and Black parents are underrepresented in the district 
school sample (Table 1). Similarly, low-income households were also underrepresented in 
the voucher sample (30%) compared to the voucher population (56%). While the authors 
noted that they were not claiming their samples are “wholly representative of the corre- 
sponding populations of parents in each of the school sectors,”? they should have addressed 
this issue explicitly as well as any possible implications for their findings. 


Table 1: Race/Ethnicity of Analysis Sample Compared to Voucher and Public School 
Populations 


Voucher District All Public 
Population School School 
Sample Students 
(N=34,299) | (N=1061) | (N=1,049,292) 


White 82% 


2 


Sources: see endnotes 7 and 8 
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Why are parents satisfied with their children’s current schools? 


While the report’s measure of parent satisfaction is based on parents’ overall assessment 
of their children’s schools, many studies of parent satisfaction ask parents how satisfied 
they were with specific features of their children’s schools such as the curriculum, teachers, 
school climate, safety, and discipline.*® Instead, the survey items focused on school qualities 
in the context of a question asking parents why they chose their children’s schools, which 
is not unrelated but a separate issue. As a result, the report does not provide insight into 
the factors that underlie parents’ satisfaction with their children’s current schools and how 
these might vary within and across sectors. 


Inadequate statistical reporting 


In the body of the report, contrasts between pairs of subgroups are often described as “sta- 
tistically significant” based on the use of z-tests to assess differences in the proportions of 
each subgroup that selected a response category. Yet this procedure is not explained in the 
text of the report but is relegated to a footnote in the tables presented in the appendix. While 
the technique is statistically correct, the z-test is an unconventional method of analyzing 
Likert items because it is focused on a single response category within a question. A more 
appropriate method would be to calculate a chi-square statistic for contingency tables that 
contain the full set of responses on a survey question for the subgroups being compared. In 
addition, the appendix does not provide all of the statistical analyses discussed in the body 
of the report. For example, the summary at the top of Figure 13 states that private school 
choice parents are “more likely to say academics are a choosing factor” compared to public 
school parents, but these results are not provided in the appendix." It is not surprising that 
the report concluded that private school choice parents are more likely to choose academ- 
ics when the survey question used in the analysis included a category specifically targeted 
to and selected by a substantial proportion of district school parents: “This is my assigned 
neighborhood school.” In this context, the use of any comparison of is problematic because 
it does not consider the broader context of the question and the full array of responses par- 
ents were asked to choose from. In addition, selecting .10 as the threshold for statistical sig- 
nificance rather than .05, which is more commonly used, inflated the number of statistically 
significant findings. While the text described the comparisons that were statistically signifi- 
cant at .10 as “marginal” differences, this distinction may not be clear to the lay reader. 


VI. Review of the Validity of the Findings and Conclusions 


The methodological issues described above raise questions about the validity of the findings. 
The supplemental information about the survey suggests additional questions.'’* When the 
survey was administered, parents were asked two screening questions: a) if they partici- 
pated in the private school choice programs, and b) how many children they had attending 
schools in other sectors. Some parents indicated that they had children enrolled in multiple 
sectors (p. 3). In the questions that followed, parents were asked to answer the survey for 
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their oldest children, suggesting that parents might enroll their children in schools across 
programs and sectors. If a substantial number of parents are utilizing multiple sectors, then 
this phenomenon needs to be further explored, particularly in settings like Indiana where 
parents have multiple options for public and private school choice. We do not know how 
many families are multi-sector families, or the extent to which parents’ experiences in mul- 
tiple sectors shape their perceptions of their children’s schools. 


Further, in the final set of analyses, 39% of the district school parents indicated that they 
switched schools. Most of these parents (77%) reported that their children’s previous school 
was a district school. This indicates that a substantial proportion of district school parents 
are engaging in school choice within the district public school sector. Yet this is not high- 
lighted as a form of school choice even though the report’s introduction briefly discussed 
magnet schools and interdistrict choice as forms of public school choice that are available in 
Indiana. Given the size of this group (548 or 16% of the parents in the analysis sample), it 
should have been included in the analysis as a sixth group. Prior studies, including a study 
cited in the literature review, have suggested that within the group of district school parents, 
active choosers, or parents who chose their children’s schools, tend to be more satisfied than 
parents whose children attend their assigned neighborhood schools.'3 However, Jochim et 
al. (2014) found no difference in parents’ satisfaction between parents who attended their 
assigned schools and parents who attended public schools of choice (e.g., magnet schools 
and charter schools).'4 


Finally, it is unclear why some findings are highlighted over others, particularly in the Ex- 
ecutive Summary. For example, the finding that non-choice private school parents are more 
likely to be satisfied with their children’s schools than district school parents was empha- 
sized rather than the finding that 74% or more of all parents are satisfied with their chil- 
dren’s schools. Likewise, the Executive Summary also highlighted differences between both 
types of private school choice parents and district school parents on two goals for their chil- 
dren, patriotism and being accepted at a top-tier college, when the charter school parents’ 
responses were almost the same as the district school parents. In these and other areas, the 
interpretation of findings tended to overemphasize differences between the district school 
parents and private sector parents and underemphasized other similarities that were evi- 
dent in the tables and figures. 


VII. Usefulness of the Report for Educational 
Policy and Practice 


While the report provided some interesting information about the parents participating in 
Indiana’s wide range of public and private school choice programs, it provided few new 
insights for educational policy and practice. The high rates of satisfaction among all five 
groups of parents confirms the findings of prior studies, but we know little about the specif- 
ic features of schools that parents value in their decision to remain at a school for the long 
term, or the areas that parents feel may need improvement. The report also failed to fully 
address its own questions about parents’ motivation for choosing schools, why parents are 
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satisfied with their children’s schools, and parents’ educational goals. If the policy goal is to 
increase enrollment in or provide political support for private school choice programs, the 
attractive layout and extensive use of colorful graphs may be effective. While organized us- 
ing the format of a research study, the report’s failure to ground its analyses in the broader 
research literature, employ conventional sampling techniques, and provide relevant statis- 
tical details renders the report of little value for advancing educational policy and practice. 


However, two understated findings from the report bear emphasizing. The Indiana parents 
surveyed here tended to largely agree on their educational goals for their children. A major- 
ity of all parents want their children to develop critical thinking skills and be prepared for 
college. At the same time, substantial proportions of Indiana’s public sector families—the 
largest constituency of K-12 parents—reported that they did not participate in private school 
choice programs because they are happy with their current schools and want to support pub- 
lic schools. Indiana policymakers need to focus how to support and improve public schools 
so that all families have the option to attend high-quality schools that meet their families’ 
educational needs and goals. 
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